<html>
<head><meta charset="utf-8"><title>concerns · t-compiler/wg-parallel-rustc · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/index.html">t-compiler/wg-parallel-rustc</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html">concerns</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="165137292"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165137292" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> njn <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165137292">(May 08 2019 at 06:00)</a>:</h4>
<p>One thing that worries me about the parallel compiler is that it will make profiling and benchmarking harder.</p>



<a name="165137322"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165137322" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> njn <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165137322">(May 08 2019 at 06:01)</a>:</h4>
<p>Instruction count is the metric with the least variance, by a long way. Currently it correlates well with wall time, which is the true metric of concern (i.e. what users observe). In a parallel compiler, that correlation will be greatly weakened.</p>



<a name="165137374"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165137374" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> njn <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165137374">(May 08 2019 at 06:02)</a>:</h4>
<p>As a result, our ability to detect small regressions (and improvements) to compiler performance will be harmed.</p>



<a name="165137409"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165137409" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> njn <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165137409">(May 08 2019 at 06:03)</a>:</h4>
<p>More generally, the use of coarse-grained parallelism in compilers is well-established and is known to work well. But I'm worried about fine-grained parallelism. The perf numbers I've seen so far have had some good improvements on some workloads, but also some drastic regressions on others.</p>



<a name="165172762"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165172762" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165172762">(May 08 2019 at 15:25)</a>:</h4>
<p>I agree that fine-grained parallelism in a compiler is a mostly uncharted field. I wonder what it would look like if we went for a more traditional approach (which usually scales better to distributed compilation).</p>



<a name="165172863"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165172863" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165172863">(May 08 2019 at 15:26)</a>:</h4>
<p>On the other hand, we should soon be in a position to do a real world evaluation of a compiled with fine-grained parallelism built in, which is pretty interesting.</p>



<a name="165173049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165173049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165173049">(May 08 2019 at 15:28)</a>:</h4>
<p>regarding profiling: the compiler will still allow being locked to only one thread, which should make the correlation between instruction count and wall-time greater again. Maybe we should keep collecting numbers for single-thread runs in order to make detection of regressions easier?</p>



<a name="165193678"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165193678" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lwshang <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165193678">(May 08 2019 at 19:34)</a>:</h4>
<p>Agree with <span class="user-mention" data-user-id="124287">@mw</span> about regression detection. We can always constraint with one thread to make the performance comparison meaningful.</p>
<p>About profiling, I'm not quite sure whether it refers to profile the compilation using rustc? Is such profiling useful?</p>



<a name="165195899"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165195899" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> njn <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165195899">(May 08 2019 at 20:02)</a>:</h4>
<p>We should measure what we ship. I'm worried that measuring single-thread runs would be misleading if we are shipping a multi-threaded compiler.</p>



<a name="165202868"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165202868" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165202868">(May 08 2019 at 21:28)</a>:</h4>
<p>Isn't multi-threaded codegen already enabled in rustc by default?<br>
Do perf runs disable it (thus decreasing variance) or not (thus not measuring what we ship)?</p>



<a name="165202931"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165202931" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165202931">(May 08 2019 at 21:29)</a>:</h4>
<p>I believe that we do not disable parallel codegen</p>



<a name="165236411"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns/near/165236411" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/concerns.html#165236411">(May 09 2019 at 09:17)</a>:</h4>
<p>We definitely should measure what we ship. But single-threaded runs could be done in addition in order to have improved regression detection.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>